Social Thermoregulation meta-analysis
Ivan Ropovik
2021-02-02
Meta-analysis results
RMA results with model-based SEs k = number of studies; sqrt in “Variance components” = tau, the standard deviation of true effects; estimate in “Model results” = naive MA estimate
RVE SEs with Satterthwaite small-sample correction Estimate based on a multilevel RE model with constant sampling correlation model (CHE - correlated hierarchical effects - working model) (Pustejovsky & Tipton, 2020; https://osf.io/preprints/metaarxiv/vyfcj/). Interpretation of naive-meta-analysis should be based on these estimates.
Prediction interval Shows the expected range of true effects in similar studies. As an approximation, in 95% of cases the true effect in a new published study can be expected to fall between PI LB and PI UB. Note that these are non-adjusted estimates. An unbiased newly conducted study will more likely fall in an interval centered around bias-adjusted estimate with a wider CI width.
Heterogeneity Tau can be interpreted as the total amount of heterogeneity in the true effects. I^2$ represents the ratio of true heterogeneity to total variance across the observed effect estimates. Estimates calculated by 2 approaches are reported. This is followed by separate estimates of between- and within-cluster heterogeneity and estimated intra-class correlation of underlying true effects.
Proportion of significant results What proportion of effects were statistically at the alpha level of .05.
ES-precision correlation Kendalls’s correlation between the ES and precision
4/3PSM Applies a permutation-based, step-function 4-parameter selection model (one-tailed p-value steps = c(.025, .5, 1)). Falls back to 3-parameter selection model if at least one of the three p-value intervals contains less than 5 p-values.
pvalue = p-value testing H0 that the effect is zero. ciLB and ciUB are lower and upper bound of the CI. k = number of studies. steps = 3 means that the 4PSM was applied, 2 means that the 3PSM was applied. For this meta-analysis, we applied 3-parameter selection model by default as there were only 11 independent effects in the opposite direction overall (6%), causing the estimates to be unstable across iterations.
PET-PEESE Estimated effect size of an infinitely precise study. Using 4/3PSM as the conditional estimator instead of PET (can be changed to PET). If the PET-PEESE estimate is in the opposite direction, the effect can be regarded nil. By default (can be changed to PET), the function employs a modified sample-size based estimator (see https://www.jepusto.com/pet-peese-performance/). It also uses the same RVE sandwich-type based estimator in a CHE (correlated hierarchical effects) working model with the identical random effects structure as the primary (naive) meta-analytic model.
We report results for both, PET and PEESE, with the first reported one being the primary (based on the conditional estimator).
WAAP-WLS The combined WAAP-WLS estimator (weighted average of the adequately powered - weighted least squares) tries to identify studies that are adequately powered to detect the meta-analytic effect. If there is less than two such studies, the method falls back to the WLS estimator (Stanley & Doucouliagos, 2015). If there are at least two adequately powered studies, WAAP returns a WLS estimate based on effects from only those studies.
type = 1: WAAP estimate, 2: WLS estimate. kAdequate = number of adequately powered studies
p-uniform P-uniform* is a selection model conceptually similar to p-curve. It makes use of the fact that p-values follow a uniform distribution at the true effect size while it includes also nonsignificant effect sizes. Permutation-based new version of p-uniform method, the so-called p-uniform* (van Aert, van Assen, 2021).
p-curve Permutation-based p-curve method. Output should be pretty self-explanatory.
Power for detecting SESOI and bias-corrected parameter estimates Estimates of the statistical power for detecting a smallest effect sizes of interest equal to .20, .50, and .70 in SD units (Cohen’s d). A sort of a thought experiment, we also assumed that population true values equal the bias-corrected estimates (4/3PSM or PET-PEESE) and computed power for those.
Handling of dependencies in bias-correction methods To handle dependencies among the effects, the 4PSM, p-curve, p-uniform are implemented using a permutation-based procedure, randomly selecting only one focal effect (i.e., excluding those which were not coded as being focal) from a single study and iterating nIterations times. Lastly, the procedure selects the result with the median value of the ES estimate (4PSM, p-uniform) or median z-score of the full p-curve (p-curve).
Moderation by prior experiences in relationships
Forest plot
Funnel plot
P-curve plot
Effect type
## [1] "The compensatory vs priming effects conceptualized by the actual direction of the effect as contrast vs. assimilation"Compensatory
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 45; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.1133 0.3366 35 no study
## sigma^2.2 0.0068 0.0826 45 no study/result
##
## Test for Heterogeneity:
## Q(df = 44) = 266.6482, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.2364 0.0659 3.5884 0.0003 0.1073 0.3655 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.236 0.0659 3.59 33 0.00106 **
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.236 0.0659 33 0.102 0.37
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## -0.480 0.953
##
## $Heterogeneity
## Tau I^2
## 0.3465356 87.8083032
## Jackson's I^2 Between-cluster heterogeneity
## 92.4000000 82.8200000
## Within-cluster heterogeneity ICC
## 4.9800000 0.9400000
##
## $`Proportion of significant results`
## [1] 0.5106383
##
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.6897864
##
## $`Publication bias`$`4/3PSM`
## est se zvalue pvalue ciLB ciUB k steps
## 0.084 0.098 0.855 0.393 -0.109 0.277 35.000 2.000
##
## $`Publication bias`$`PET-PEESE`
## PET estimate se zvalue pvalue ciLB
## -0.295 0.167 -1.767 0.086 -0.634
## ciUB PEESE estimate se zvalue pvalue
## 0.045 0.032 0.097 0.328 0.745
## ciLB ciUB
## -0.165 0.229
##
## $`Publication bias`$`WAAP-WLS`
## method term estimate std.error statistic p.value conf.low conf.high
## 1 WAAP-WLS b0 0.14467 0.04490171 0.04490171 0.00239815 0.05417654 0.2351635
## type kAdequate
## 1 2 1
##
## $`Publication bias`$`p-uniform*`
## est ciLB ciUB pvalue
## -0.01100584 -0.19044259 0.17447474 0.90604429
##
## $`Publication bias`$`p-curve`
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 52
## - Total number of p<0.05 studies included into the analysis: k = 36 (69.23%)
## - Total number of studies with p<0.025: k = 25 (48.08%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.014 -6.934 0.000 -6.997 0
## Flatness test 0.459 3.037 0.999 9.525 1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 68% (49.9%-81.1%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: no
##
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## Median power for detecting a SESOI of d = .20
## "0.385"
## Median power for detecting a SESOI of d = .50
## "0.986"
## Median power for detecting a SESOI of d = .70
## "1"
## Median power for detecting PET-PEESE estimate.PET estimate
## "ES estimate in the opposite direction"
## Median power for detecting 4/3PSM estimate.est
## "0.108"Priming
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 125; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0347 0.1864 84 no study
## sigma^2.2 0.0196 0.1399 125 no study/result
##
## Test for Heterogeneity:
## Q(df = 124) = 315.2924, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.4497 0.0339 13.2675 <.0001 0.3833 0.5162 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.45 0.0339 13.3 73.9 <0.001 ***
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.45 0.0339 73.9 0.382 0.517
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## -0.019 0.918
##
## $Heterogeneity
## Tau I^2
## 0.2330976 60.9134712
## Jackson's I^2 Between-cluster heterogeneity
## 76.8600000 38.9600000
## Within-cluster heterogeneity ICC
## 21.9600000 0.6400000
##
## $`Proportion of significant results`
## [1] 0.6136364
##
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.5434097
##
## $`Publication bias`$`4/3PSM`
## est se zvalue pvalue ciLB ciUB k steps
## 0.248 0.052 4.780 0.000 0.147 0.350 83.000 2.000
##
## $`Publication bias`$`PET-PEESE`
## PEESE estimate se zvalue pvalue ciLB
## 0.204 0.056 3.656 0.000 0.093
## ciUB PET estimate se zvalue pvalue
## 0.315 -0.049 0.060 -0.810 0.421
## ciLB ciUB
## -0.169 0.071
##
## $`Publication bias`$`WAAP-WLS`
## method term estimate std.error statistic p.value conf.low
## 1 WAAP-WLS b0 0.06033676 0.03493944 0.03493944 0.1349326 -0.02515696
## conf.high type kAdequate
## 1 0.1458305 1 7
##
## $`Publication bias`$`p-uniform*`
## est ciLB ciUB pvalue
## 0.28237593329 0.17999288439 0.38133351331 0.00001748352
##
## $`Publication bias`$`p-curve`
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 116
## - Total number of p<0.05 studies included into the analysis: k = 80 (68.97%)
## - Total number of studies with p<0.025: k = 50 (43.1%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.016 -6.251 0.000 -8.037 0
## Flatness test 0.053 0.347 0.636 9.925 1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 36% (23.4%-49.7%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: no
##
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## Median power for detecting a SESOI of d = .20
## 0.202
## Median power for detecting a SESOI of d = .50
## 0.801
## Median power for detecting a SESOI of d = .70
## 0.976
## Median power for detecting PET-PEESE estimate.PEESE estimate
## 0.209
## Median power for detecting 4/3PSM estimate.est
## 0.286Comparison of effect types
Model without covariates
## $`Model results`
## $`Model results`$test
## Coef. Estimate SE t-stat p-val (z) Sig.
## 1 factor(effectCompPriming)1 0.211 0.0644 3.27 0.00109 **
## 2 factor(effectCompPriming)2 0.485 0.0321 15.13 < 0.001 ***
##
## $`Model results`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 factor(effectCompPriming)1 0.211 0.0644 Inf 0.0842 0.337
## 2 factor(effectCompPriming)2 0.485 0.0321 Inf 0.4221 0.548
##
##
## $`RVE Wald test`
## test Fstat df_num df_denom p_val sig
## HTZ 15.3 1 35.3 <0.001 ***Model with covariates
Controlling for design-related factors that are prognostic w.r.t. the effect sizes (i.e., might vary across moderator categories), namely rct, published, sourceTargetDirectionality, and studentSample.
## $`Model results`
## $`Model results`$test
## Coef. Estimate SE t-stat p-val (z) Sig.
## 1 factor(effectCompPriming)1 0.00226 0.0900 0.0251 0.9799
## 2 factor(effectCompPriming)2 0.19823 0.1046 1.8951 0.0581 .
## 3 rct 0.14418 0.0747 1.9312 0.0535 .
## 4 published 0.15282 0.0730 2.0924 0.0364 *
## 5 sourceTargetDirectionality_reconcil -0.03291 0.0806 -0.4083 0.6831
## 6 studentSample 0.06213 0.0751 0.8269 0.4083
##
## $`Model results`$CIs
## Coef Estimate SE d.f. Lower 95% CI
## 1 factor(effectCompPriming)1 0.00226 0.0900 Inf -0.17419
## 2 factor(effectCompPriming)2 0.19823 0.1046 Inf -0.00679
## 3 rct 0.14418 0.0747 Inf -0.00215
## 4 published 0.15282 0.0730 Inf 0.00967
## 5 sourceTargetDirectionality_reconcil -0.03291 0.0806 Inf -0.19087
## 6 studentSample 0.06213 0.0751 Inf -0.08514
## Upper 95% CI
## 1 0.179
## 2 0.403
## 3 0.291
## 4 0.296
## 5 0.125
## 6 0.209
##
##
## $`RVE Wald test`
## test Fstat df_num df_denom p_val sig
## HTZ 9.43 1 22.3 0.00554 **## null device
## 1Plots
Contour enhanced funnel plot
Compensatory
Priming
Forest plots
Compensatory
Priming
p-curve plots
Compensatory
Priming
PET-PEESE plots
Using the sqrt(2/n) and 2/n terms instead of SE and var for PET and PEESE, respectively since modified sample-size based estimator was implemented (see https://www.jepusto.com/pet-peese-performance/).
Compensatory
Priming
## null device
## 1Mood
Results
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 17; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0000 0.0000 11 no study
## sigma^2.2 0.0000 0.0000 17 no study/result
##
## Test for Heterogeneity:
## Q(df = 16) = 7.1492, p-val = 0.9703
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.2315 0.0698 3.3145 0.0009 0.0946 0.3684 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.231 0.0448 5.17 8.9 <0.001 ***
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.231 0.0448 8.9 0.13 0.333
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## 0.131 0.332
##
## $Heterogeneity
## Tau I^2
## 0.00000295647900 0.00000001354476
## Jackson's I^2 Between-cluster heterogeneity
## 0.00000000000000 0.00000000000000
## Within-cluster heterogeneity ICC
## 0.00000000000000 1.00000000000000
##
## $`Proportion of significant results`
## [1] 0.1176471
##
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"Forest plot
Funnel plot
Overall effect results
Results
Number of iterations run equal to 200 for p-curve and 5000 for all other bias correction functions.
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 175; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0438 0.2092 116 no study
## sigma^2.2 0.0357 0.1888 175 no study/result
##
## Test for Heterogeneity:
## Q(df = 174) = 677.7377, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.4085 0.0315 12.9864 <.0001 0.3469 0.4702 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.409 0.0315 13 105 <0.001 ***
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.409 0.0315 105 0.346 0.471
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## -0.153 0.970
##
## $Heterogeneity
## Tau I^2
## 0.2818135 74.9082576
## Jackson's I^2 Between-cluster heterogeneity
## 85.1400000 41.2700000
## Within-cluster heterogeneity ICC
## 33.6400000 0.5500000
##
## $`Proportion of significant results`
## [1] 0.5923913
##
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.5796001
##
## $`Publication bias`$`4/3PSM`
## est se zvalue pvalue ciLB ciUB k steps
## 0.187 0.052 3.599 0.000 0.085 0.289 114.000 2.000
##
## $`Publication bias`$`PET-PEESE`
## PEESE estimate se zvalue pvalue ciLB
## 0.159 0.053 3.008 0.003 0.054
## ciUB PET estimate se zvalue pvalue
## 0.264 -0.138 0.072 -1.915 0.058
## ciLB ciUB
## -0.280 0.005
##
## $`Publication bias`$`WAAP-WLS`
## method term estimate std.error statistic p.value conf.low conf.high
## 1 WAAP-WLS b0 0.1493071 0.09129957 0.09129957 0.2436033 -0.2435232 0.5421375
## type kAdequate
## 1 1 3
##
## $`Publication bias`$`p-uniform*`
## est ciLB ciUB pvalue
## 0.1944982257 0.0957078307 0.2920326952 0.0005655032
##
## $`Publication bias`$`p-curve`
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 166
## - Total number of p<0.05 studies included into the analysis: k = 115 (69.28%)
## - Total number of studies with p<0.025: k = 75 (45.18%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.001 -8.782 0.000 -10.217 0
## Flatness test 0.087 1.653 0.951 13.089 1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 45% (33.4%-55.5%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: no
##
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## Median power for detecting a SESOI of d = .20
## 0.215
## Median power for detecting a SESOI of d = .50
## 0.830
## Median power for detecting a SESOI of d = .70
## 0.983
## Median power for detecting PET-PEESE estimate.PEESE estimate
## 0.153
## Median power for detecting 4/3PSM estimate.est
## 0.193Forest plot
Funnel plot
p-curve plot
Methods
Moderator analysis
## $`Model results`
## $`Model results`$test
## Coef. Estimate SE t-stat p-val (z) Sig.
## 1 methodPhysical temperature manipulation 0.464 0.0629 7.37 < 0.001 ***
## 2 methodVisual/verbal temperature prime 0.485 0.0589 8.23 < 0.001 ***
## 3 methodOutside temperature 0.379 0.1512 2.51 0.01223 *
## 4 methodTemperature estimate as DV 0.477 0.0703 6.79 < 0.001 ***
## 5 methodSubjective warmth judgment as DV 0.299 0.1149 2.61 0.00915 **
##
## $`Model results`$CIs
## Coef Estimate SE d.f. Lower 95% CI
## 1 methodPhysical temperature manipulation 0.464 0.0629 Inf 0.3406
## 2 methodVisual/verbal temperature prime 0.485 0.0589 Inf 0.3693
## 3 methodOutside temperature 0.379 0.1512 Inf 0.0825
## 4 methodTemperature estimate as DV 0.477 0.0703 Inf 0.3396
## 5 methodSubjective warmth judgment as DV 0.299 0.1149 Inf 0.0743
## Upper 95% CI
## 1 0.587
## 2 0.600
## 3 0.675
## 4 0.615
## 5 0.525
##
##
## $`RVE Wald test`
## test Fstat df_num df_denom p_val sig
## HTZ 0.682 4 17.5 0.614Results for different methods
Leaving out the Core temperature measurement and Skin temperature measurement, since k is too low
Meta-analysis results
#’ Brief results
## $`Physical temperature manipulation`
## k g [95% CI] SE Tau
## 83 0.48 [0.36, 0.59] 0.06 0.34
## I^2
## 69%
##
## $`Visual/verbal temperature prime`
## k g [95% CI] SE Tau
## 23 0.44 [0.35, 0.53] 0.04 0.15
## I^2
## 38%
##
## $`Outside temperature`
## k g [95% CI] SE Tau
## 13 0.16 [-0.02, 0.34] 0.06 0.1
## I^2
## 44%
##
## $`Temperature estimate as DV`
## k g [95% CI] SE Tau
## 25 0.39 [0.27, 0.52] 0.06 0.19
## I^2
## 60%
##
## $`Subjective warmth judgment as DV`
## k g [95% CI] SE Tau
## 14 0.28 [0.03, 0.53] 0.11 0.37
## I^2
## 93%Physical temperature manipulation
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 82; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.1003 0.3167 51 no study
## sigma^2.2 0.0153 0.1238 82 no study/result
##
## Test for Heterogeneity:
## Q(df = 81) = 339.5264, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.4779 0.0575 8.3075 <.0001 0.3652 0.5907 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.478 0.0576 8.3 48.2 <0.001 ***
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.478 0.0576 48.2 0.362 0.594
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## -0.215 1.171
##
## $Heterogeneity
## Tau I^2
## 0.3400426 69.0555528
## Jackson's I^2 Between-cluster heterogeneity
## 80.9300000 59.9000000
## Within-cluster heterogeneity ICC
## 9.1500000 0.8700000
##
## $`Proportion of significant results`
## [1] 0.6746988
##
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"Visual/verbal temperature prime
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 23; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0000 0.0000 14 no study
## sigma^2.2 0.0232 0.1524 23 no study/result
##
## Test for Heterogeneity:
## Q(df = 22) = 38.5260, p-val = 0.0160
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.4398 0.0600 7.3355 <.0001 0.3223 0.5573 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.44 0.0411 10.7 11.4 <0.001 ***
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.44 0.0411 11.4 0.35 0.53
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## 0.099 0.781
##
## $Heterogeneity
## Tau I^2
## 0.1523991 37.8013636
## Jackson's I^2 Between-cluster heterogeneity
## 13.6800000 0.0000000
## Within-cluster heterogeneity ICC
## 37.8000000 0.0000000
##
## $`Proportion of significant results`
## [1] 0.6956522
##
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"Outside temperature
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 8; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0077 0.0875 6 no study
## sigma^2.2 0.0028 0.0525 8 no study/result
##
## Test for Heterogeneity:
## Q(df = 7) = 14.2050, p-val = 0.0477
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.1607 0.0642 2.5034 0.0123 0.0349 0.2864 *
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.161 0.0634 2.54 3.63 0.0705 .
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.161 0.0634 3.63 -0.0225 0.344
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## -0.147 0.468
##
## $Heterogeneity
## Tau I^2
## 0.1020439 44.1844740
## Jackson's I^2 Between-cluster heterogeneity
## 56.9100000 32.5100000
## Within-cluster heterogeneity ICC
## 11.6700000 0.7400000
##
## $`Proportion of significant results`
## [1] 0.1538462
##
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"Temperature estimate as DV
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 23; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0378 0.1945 21 no study
## sigma^2.2 0.0000 0.0000 23 no study/result
##
## Test for Heterogeneity:
## Q(df = 22) = 49.5679, p-val = 0.0007
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.3931 0.0599 6.5631 <.0001 0.2757 0.5104 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.393 0.0601 6.54 17 <0.001 ***
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.393 0.0601 17 0.266 0.52
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## -0.031 0.817
##
## $Heterogeneity
## Tau I^2
## 0.1945094 59.9035594
## Jackson's I^2 Between-cluster heterogeneity
## 71.0500000 59.9000000
## Within-cluster heterogeneity ICC
## 0.0000000 1.0000000
##
## $`Proportion of significant results`
## [1] 0.72
##
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"Subjective warmth judgment as DV
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 13; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.1253 0.3539 12 no study
## sigma^2.2 0.0089 0.0942 13 no study/result
##
## Test for Heterogeneity:
## Q(df = 12) = 69.2488, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.2810 0.1153 2.4362 0.0148 0.0549 0.5070 *
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.281 0.115 2.45 10.7 0.033 *
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.281 0.115 10.7 0.0274 0.535
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## -0.564 1.126
##
## $Heterogeneity
## Tau I^2
## 0.3662546 92.6858591
## Jackson's I^2 Between-cluster heterogeneity
## 95.7600000 86.5600000
## Within-cluster heterogeneity ICC
## 6.1300000 0.9300000
##
## $`Proportion of significant results`
## [1] 0.5714286
##
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"Bias correction results
Physical temperature manipulation
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.5226101
##
## $`Publication bias`$`4/3PSM`
## est se zvalue pvalue ciLB ciUB k steps
## 0.151 0.094 1.603 0.109 -0.034 0.336 51.000 2.000
##
## $`Publication bias`$`PET-PEESE`
## PET estimate se zvalue pvalue ciLB
## -0.254 0.225 -1.128 0.265 -0.705
## ciUB PEESE estimate se zvalue pvalue
## 0.198 0.190 0.133 1.424 0.161
## ciLB ciUB
## -0.078 0.457
##
## $`Publication bias`$`WAAP-WLS`
## method term estimate std.error statistic p.value conf.low conf.high
## 1 WAAP-WLS b0 -0.4364618 0.251926 0.251926 0.2253231 -1.520412 0.6474885
## type kAdequate
## 1 1 3
##
## $`Publication bias`$`p-uniform*`
## est ciLB ciUB pvalue
## 0.06957033 -0.13193200 0.27026087 0.52185334
##
## $`Publication bias`$`p-curve`
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 82
## - Total number of p<0.05 studies included into the analysis: k = 58 (70.73%)
## - Total number of studies with p<0.025: k = 33 (40.24%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.179 -6.720 0.000 -8.942 0
## Flatness test 0.013 1.791 0.963 10.255 1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 50% (34.7%-65%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: no
##
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## Median power for detecting a SESOI of d = .20
## "0.19"
## Median power for detecting a SESOI of d = .50
## "0.768"
## Median power for detecting a SESOI of d = .70
## "0.965"
## Median power for detecting PET-PEESE estimate.PET estimate
## "ES estimate in the opposite direction"
## Median power for detecting 4/3PSM estimate.est
## "0.129"Visual/verbal temperature prime
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.3287924
##
## $`Publication bias`$`4/3PSM`
## est se zvalue pvalue ciLB ciUB k steps
## 0.271 0.080 3.400 0.001 0.115 0.428 14.000 2.000
##
## $`Publication bias`$`PET-PEESE`
## PEESE estimate se zvalue pvalue ciLB
## 0.292 0.110 2.662 0.021 0.053
## ciUB PET estimate se zvalue pvalue
## 0.532 0.182 0.207 0.878 0.397
## ciLB ciUB
## -0.270 0.634
##
## $`Publication bias`$`WAAP-WLS`
## method term estimate std.error statistic p.value conf.low conf.high
## 1 WAAP-WLS b0 0.3037822 0.03003057 0.03003057 0.009631472 0.1745711 0.4329934
## type kAdequate
## 1 1 3
##
## $`Publication bias`$`p-uniform*`
## est ciLB ciUB pvalue
## 0.31186560 0.09457384 0.49685638 0.23877258
##
## $`Publication bias`$`p-curve`
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 23
## - Total number of p<0.05 studies included into the analysis: k = 19 (82.61%)
## - Total number of studies with p<0.025: k = 14 (60.87%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.032 -4.527 0.000 -4.793 0
## Flatness test 0.670 1.450 0.926 5.666 1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 56% (30.3%-77.3%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: no
##
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## Median power for detecting a SESOI of d = .20
## 0.202
## Median power for detecting a SESOI of d = .50
## 0.801
## Median power for detecting a SESOI of d = .70
## 0.976
## Median power for detecting PET-PEESE estimate.PEESE estimate
## 0.374
## Median power for detecting 4/3PSM estimate.est
## 0.331Temperature estimate as DV
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.7500059
##
## $`Publication bias`$`4/3PSM`
## est se zvalue pvalue ciLB ciUB k steps
## 0.046 0.058 0.779 0.436 -0.069 0.160 21.000 2.000
##
## $`Publication bias`$`PET-PEESE`
## PET estimate se zvalue pvalue ciLB
## -0.086 0.091 -0.942 0.358 -0.277
## ciUB PEESE estimate se zvalue pvalue
## 0.105 0.108 0.062 1.742 0.098
## ciLB ciUB
## -0.022 0.238
##
## $`Publication bias`$`WAAP-WLS`
## method term estimate std.error statistic p.value conf.low conf.high
## 1 WAAP-WLS b0 0.04964282 0.1242567 0.1242567 0.7580266 -1.529189 1.628474
## type kAdequate
## 1 1 2
##
## $`Publication bias`$`p-uniform*`
## est ciLB ciUB pvalue
## 0.0556670 -0.1507642 0.2412312 0.6814799
##
## $`Publication bias`$`p-curve`
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 24
## - Total number of p<0.05 studies included into the analysis: k = 18 (75%)
## - Total number of studies with p<0.025: k = 10 (41.67%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.407 -0.951 0.171 -1.939 0.026
## Flatness test 0.112 -1.590 0.056 3.610 1.000
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 10% (5%-34.4%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: no
##
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## Median power for detecting a SESOI of d = .20
## "0.202"
## Median power for detecting a SESOI of d = .50
## "0.801"
## Median power for detecting a SESOI of d = .70
## "0.976"
## Median power for detecting PET-PEESE estimate.PET estimate
## "ES estimate in the opposite direction"
## Median power for detecting 4/3PSM estimate.est
## "0.058"Funnel plots
Forest plots
Categories
Leaving out the Robotics and Neural Mechanisms, since k is too low
Results for different categories
Meta-analysis results
Brief results
## $Emotion
## k g [95% CI] SE Tau
## 24 0.39 [0.31, 0.46] 0.03 0.15
## I^2
## 54%
##
## $Interpersonal
## k g [95% CI] SE Tau
## 76 0.36 [0.26, 0.46] 0.05 0.31
## I^2
## 78%
##
## $`Person perception`
## k g [95% CI] SE Tau
## 39 0.41 [0.23, 0.58] 0.08 0.33
## I^2
## 81%
##
## $`Group processes`
## k g [95% CI] SE Tau
## 12 0.62 [0.38, 0.85] 0.09 0
## I^2
## 0%
##
## $`Moral judgment`
## k g [95% CI] SE Tau
## 6 0.49 [-0.12, 1.1] 0.11 0.03
## I^2
## 2%
##
## $`Self-regulation`
## k g [95% CI] SE Tau
## 26 0.32 [0.17, 0.47] 0.07 0.27
## I^2
## 76%
##
## $`Cognitive processes`
## k g [95% CI] SE Tau
## 36 0.56 [0.46, 0.66] 0.05 0.12
## I^2
## 20%
##
## $`Economic decision-making`
## k g [95% CI] SE Tau
## 43 0.44 [0.28, 0.59] 0.07 0.31
## I^2
## 70%Emotion
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 23; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0000 0.0000 19 no study
## sigma^2.2 0.0233 0.1526 23 no study/result
##
## Test for Heterogeneity:
## Q(df = 22) = 44.6519, p-val = 0.0029
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.3859 0.0506 7.6308 <.0001 0.2868 0.4850 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.386 0.0343 11.2 15.3 <0.001 ***
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.386 0.0343 15.3 0.313 0.459
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## 0.057 0.715
##
## $Heterogeneity
## Tau I^2
## 0.1526369 54.0804386
## Jackson's I^2 Between-cluster heterogeneity
## 65.7500000 0.0000000
## Within-cluster heterogeneity ICC
## 54.0800000 0.0000000
##
## $`Proportion of significant results`
## [1] 0.7083333
##
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"Interpersonal
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 75; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0503 0.2242 56 no study
## sigma^2.2 0.0488 0.2208 75 no study/result
##
## Test for Heterogeneity:
## Q(df = 74) = 364.3079, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.3594 0.0498 7.2141 <.0001 0.2617 0.4570 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.359 0.0499 7.21 51.7 <0.001 ***
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.359 0.0499 51.7 0.259 0.459
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## -0.279 0.998
##
## $Heterogeneity
## Tau I^2
## 0.3146483 77.6948925
## Jackson's I^2 Between-cluster heterogeneity
## 85.2200000 39.4400000
## Within-cluster heterogeneity ICC
## 38.2600000 0.5100000
##
## $`Proportion of significant results`
## [1] 0.5789474
##
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"Person perception
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 36; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0765 0.2766 21 no study
## sigma^2.2 0.0337 0.1835 36 no study/result
##
## Test for Heterogeneity:
## Q(df = 35) = 133.8868, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.4075 0.0829 4.9175 <.0001 0.2451 0.5698 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.407 0.0828 4.92 18.3 <0.001 ***
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.407 0.0828 18.3 0.234 0.581
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## -0.306 1.121
##
## $Heterogeneity
## Tau I^2
## 0.3319022 81.0462124
## Jackson's I^2 Between-cluster heterogeneity
## 90.0500000 56.2800000
## Within-cluster heterogeneity ICC
## 24.7700000 0.6900000
##
## $`Proportion of significant results`
## [1] 0.4358974
##
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"Group processes
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 11; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0000 0.0000 7 no study
## sigma^2.2 0.0000 0.0000 11 no study/result
##
## Test for Heterogeneity:
## Q(df = 10) = 8.8815, p-val = 0.5434
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.6152 0.1079 5.7024 <.0001 0.4037 0.8266 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.615 0.094 6.55 5.3 <0.001 ***
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.615 0.094 5.3 0.378 0.853
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## 0.386 0.845
##
## $Heterogeneity
## Tau I^2
## 0.00000462993551 0.00000002010181
## Jackson's I^2 Between-cluster heterogeneity
## 0.00000000000000 0.00000000000000
## Within-cluster heterogeneity ICC
## 0.00000000000000 0.83000000000000
##
## $`Proportion of significant results`
## [1] 0.8333333
##
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"Moral judgment
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 6; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0000 0.0000 3 no study
## sigma^2.2 0.0011 0.0329 6 no study/result
##
## Test for Heterogeneity:
## Q(df = 5) = 4.6596, p-val = 0.4588
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.4878 0.1076 4.5335 <.0001 0.2769 0.6987 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.488 0.111 4.41 1.61 0.0701 .
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.488 0.111 1.61 -0.12 1.1
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## 0.027 0.948
##
## $Heterogeneity
## Tau I^2
## 0.03292223 2.34692161
## Jackson's I^2 Between-cluster heterogeneity
## 25.18000000 0.00000000
## Within-cluster heterogeneity ICC
## 2.35000000 0.00000000
##
## $`Proportion of significant results`
## [1] NA
##
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"Self-regulation
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 24; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0312 0.1765 21 no study
## sigma^2.2 0.0436 0.2089 24 no study/result
##
## Test for Heterogeneity:
## Q(df = 23) = 90.2528, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.3242 0.0716 4.5298 <.0001 0.1839 0.4645 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.324 0.0714 4.54 18.7 <0.001 ***
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.324 0.0714 18.7 0.175 0.474
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## -0.265 0.914
##
## $Heterogeneity
## Tau I^2
## 0.2734749 76.4112635
## Jackson's I^2 Between-cluster heterogeneity
## 84.2000000 31.8400000
## Within-cluster heterogeneity ICC
## 44.5800000 0.4200000
##
## $`Proportion of significant results`
## [1] 0.5384615
##
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"Cognitive processes
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 35; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0000 0.0000 29 no study
## sigma^2.2 0.0143 0.1194 35 no study/result
##
## Test for Heterogeneity:
## Q(df = 34) = 40.8129, p-val = 0.1959
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.5566 0.0485 11.4785 <.0001 0.4616 0.6517 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.557 0.0478 11.6 23.2 <0.001 ***
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.557 0.0478 23.2 0.458 0.655
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## 0.293 0.820
##
## $Heterogeneity
## Tau I^2
## 0.1194099 19.7930556
## Jackson's I^2 Between-cluster heterogeneity
## 26.6100000 0.0000000
## Within-cluster heterogeneity ICC
## 19.7900000 0.0000000
##
## $`Proportion of significant results`
## [1] 0.75
##
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"Economic decision-making
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 38; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0671 0.2590 25 no study
## sigma^2.2 0.0306 0.1750 38 no study/result
##
## Test for Heterogeneity:
## Q(df = 37) = 117.7860, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.4361 0.0743 5.8697 <.0001 0.2905 0.5818 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.436 0.0743 5.87 22.7 <0.001 ***
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.436 0.0743 22.7 0.282 0.59
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## -0.227 1.099
##
## $Heterogeneity
## Tau I^2
## 0.312574 70.182750
## Jackson's I^2 Between-cluster heterogeneity
## 81.650000 48.180000
## Within-cluster heterogeneity ICC
## 22.000000 0.690000
##
## $`Proportion of significant results`
## [1] 0.5581395
##
## $`Publication bias`
## [1] "Publication bias corrections not carried out"
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## [1] "Power for detecting bias-corrected parameter estimates not computed"## 3PSM est [95% CI] 3PSM.pvalue PET-PEESE est [95% CI]
## 0.23 [-0.01, 0.48] 0.061 0.24 [-0.04, 0.52]
## PET-PEESE.pvalue
## 0.086## $Emotion
## k g [95% CI] SE Tau
## 24 0.39 [0.31, 0.46] 0.03 0.15
## I^2
## 54%
##
## $Interpersonal
## k g [95% CI] SE Tau
## 76 0.36 [0.26, 0.46] 0.05 0.31
## I^2
## 78%
##
## $`Person perception`
## k g [95% CI] SE Tau
## 39 0.41 [0.23, 0.58] 0.08 0.33
## I^2
## 81%
##
## $`Group processes`
## k g [95% CI] SE Tau
## 12 0.62 [0.38, 0.85] 0.09 0
## I^2
## 0%
##
## $`Moral judgment`
## k g [95% CI] SE Tau
## 6 0.49 [-0.12, 1.1] 0.11 0.03
## I^2
## 2%
##
## $`Self-regulation`
## k g [95% CI] SE Tau
## 26 0.32 [0.17, 0.47] 0.07 0.27
## I^2
## 76%
##
## $`Cognitive processes`
## k g [95% CI] SE Tau
## 36 0.56 [0.46, 0.66] 0.05 0.12
## I^2
## 20%
##
## $`Economic decision-making`
## k g [95% CI] SE Tau
## 43 0.44 [0.28, 0.59] 0.07 0.31
## I^2
## 70%Bias correction results
Emotion
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.2669407
##
## $`Publication bias`$`4/3PSM`
## est se zvalue pvalue ciLB ciUB k steps
## 0.260 0.028 9.171 0.000 0.204 0.315 19.000 2.000
##
## $`Publication bias`$`PET-PEESE`
## PEESE estimate se zvalue pvalue ciLB
## 0.353 0.050 7.110 0.000 0.249
## ciUB PET estimate se zvalue pvalue
## 0.458 0.301 0.064 4.724 0.000
## ciLB ciUB
## 0.167 0.436
##
## $`Publication bias`$`WAAP-WLS`
## method term estimate std.error statistic p.value conf.low
## 1 WAAP-WLS b0 0.3105771 0.02925429 0.02925429 0.000000000402088 0.2499074
## conf.high type kAdequate
## 1 0.3712468 2 1
##
## $`Publication bias`$`p-uniform*`
## est ciLB ciUB pvalue
## 0.30896207 0.19689270 0.41491368 0.01497127
##
## $`Publication bias`$`p-curve`
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 31
## - Total number of p<0.05 studies included into the analysis: k = 23 (74.19%)
## - Total number of studies with p<0.025: k = 14 (45.16%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.202 -4.994 0.000 -6.551 0
## Flatness test 0.185 1.740 0.959 7.135 1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 59% (34.7%-78%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: no
##
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## Median power for detecting a SESOI of d = .20
## 0.331
## Median power for detecting a SESOI of d = .50
## 0.968
## Median power for detecting a SESOI of d = .70
## 1.000
## Median power for detecting PET-PEESE estimate.PEESE estimate
## 0.767
## Median power for detecting 4/3PSM estimate.est
## 0.508Interpersonal
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.5905207
##
## $`Publication bias`$`4/3PSM`
## est se zvalue pvalue ciLB ciUB k steps
## 0.100 0.078 1.295 0.195 -0.052 0.252 56.000 2.000
##
## $`Publication bias`$`PET-PEESE`
## PET estimate se zvalue pvalue ciLB
## -0.215 0.107 -2.000 0.051 -0.430
## ciUB PEESE estimate se zvalue pvalue
## 0.000 0.098 0.077 1.276 0.207
## ciLB ciUB
## -0.056 0.252
##
## $`Publication bias`$`WAAP-WLS`
## method term estimate std.error statistic p.value conf.low
## 1 WAAP-WLS b0 0.1388678 0.04068096 0.04068096 0.001043176 0.05780919
## conf.high type kAdequate
## 1 0.2199264 2 1
##
## $`Publication bias`$`p-uniform*`
## est ciLB ciUB pvalue
## 0.07458773 -0.08100454 0.23262782 0.35226835
##
## $`Publication bias`$`p-curve`
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 87
## - Total number of p<0.05 studies included into the analysis: k = 56 (64.37%)
## - Total number of studies with p<0.025: k = 31 (35.63%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.252 -4.513 0.000 -6.905 0
## Flatness test 0.008 -0.084 0.466 8.829 1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 32% (17.9%-49.2%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: no
##
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## Median power for detecting a SESOI of d = .20
## "0.205"
## Median power for detecting a SESOI of d = .50
## "0.808"
## Median power for detecting a SESOI of d = .70
## "0.977"
## Median power for detecting PET-PEESE estimate.PET estimate
## "ES estimate in the opposite direction"
## Median power for detecting 4/3PSM estimate.est
## "0.087"Person perception
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.6107588
##
## $`Publication bias`$`4/3PSM`
## est se zvalue pvalue ciLB ciUB k steps
## 0.442 0.125 3.528 0.000 0.196 0.687 21.000 2.000
##
## $`Publication bias`$`PET-PEESE`
## PEESE estimate se zvalue pvalue ciLB
## 0.060 0.050 1.181 0.252 -0.046
## ciUB PET estimate se zvalue pvalue
## 0.165 -0.236 0.064 -3.710 0.001
## ciLB ciUB
## -0.369 -0.103
##
## $`Publication bias`$`WAAP-WLS`
## method term estimate std.error statistic p.value conf.low
## 1 WAAP-WLS b0 0.1963766 0.05084897 0.05084897 0.0004645318 0.09314768
## conf.high type kAdequate
## 1 0.2996055 2 0
##
## $`Publication bias`$`p-uniform*`
## est ciLB ciUB pvalue
## 0.3497897 0.1311900 0.5620256 0.0055566
##
## $`Publication bias`$`p-curve`
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 27
## - Total number of p<0.05 studies included into the analysis: k = 18 (66.67%)
## - Total number of studies with p<0.025: k = 12 (44.44%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.119 -5.415 0.000 -6.263 0
## Flatness test 0.413 2.363 0.991 6.659 1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 69% (44.9%-85.8%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: no
##
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## Median power for detecting a SESOI of d = .20
## 0.253
## Median power for detecting a SESOI of d = .50
## 0.899
## Median power for detecting a SESOI of d = .70
## 0.995
## Median power for detecting PET-PEESE estimate.PEESE estimate
## 0.067
## Median power for detecting 4/3PSM estimate.est
## 0.816Self-regulation
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.6350534
##
## $`Publication bias`$`4/3PSM`
## est se zvalue pvalue ciLB ciUB k steps
## 0.068 0.114 0.600 0.548 -0.155 0.292 21.000 2.000
##
## $`Publication bias`$`PET-PEESE`
## PET estimate se zvalue pvalue ciLB
## -0.318 0.101 -3.148 0.005 -0.530
## ciUB PEESE estimate se zvalue pvalue
## -0.107 -0.003 0.056 -0.057 0.955
## ciLB ciUB
## -0.120 0.114
##
## $`Publication bias`$`WAAP-WLS`
## method term estimate std.error statistic p.value conf.low conf.high
## 1 WAAP-WLS b0 0.1496 0.05771728 0.05771728 0.01630102 0.03020272 0.2689973
## type kAdequate
## 1 2 1
##
## $`Publication bias`$`p-uniform*`
## est ciLB ciUB pvalue
## 0.12567927 -0.05613884 0.31311091 0.18274946
##
## $`Publication bias`$`p-curve`
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 35
## - Total number of p<0.05 studies included into the analysis: k = 24 (68.57%)
## - Total number of studies with p<0.025: k = 15 (42.86%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.154 -2.474 0.007 -2.071 0.019
## Flatness test 0.225 -0.642 0.260 4.357 1.000
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 24% (8.1%-48.8%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: no
##
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## Median power for detecting a SESOI of d = .20
## "0.396"
## Median power for detecting a SESOI of d = .50
## "0.989"
## Median power for detecting a SESOI of d = .70
## "1"
## Median power for detecting PET-PEESE estimate.PET estimate
## "ES estimate in the opposite direction"
## Median power for detecting 4/3PSM estimate.est
## "0.089"Cognitive processes
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.3800724
##
## $`Publication bias`$`4/3PSM`
## est se zvalue pvalue ciLB ciUB k steps
## 0.344 0.129 2.678 0.007 0.092 0.596 28.000 2.000
##
## $`Publication bias`$`PET-PEESE`
## PEESE estimate se zvalue pvalue ciLB
## 0.405 0.083 4.859 0.000 0.234
## ciUB PET estimate se zvalue pvalue
## 0.576 0.218 0.146 1.490 0.148
## ciLB ciUB
## -0.082 0.519
##
## $`Publication bias`$`WAAP-WLS`
## method term estimate std.error statistic p.value conf.low conf.high
## 1 WAAP-WLS b0 0.4210579 0.1142019 0.1142019 0.02107484 0.1039825 0.7381332
## type kAdequate
## 1 1 5
##
## $`Publication bias`$`p-uniform*`
## est ciLB ciUB pvalue
## 0.24115193 0.01326872 0.44907076 0.21851476
##
## $`Publication bias`$`p-curve`
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 51
## - Total number of p<0.05 studies included into the analysis: k = 38 (74.51%)
## - Total number of studies with p<0.025: k = 25 (49.02%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.036 -5.536 0.00 -6.390 0
## Flatness test 0.273 1.403 0.92 7.798 1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 50% (30.6%-67.2%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: no
##
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## Median power for detecting a SESOI of d = .20
## 0.197
## Median power for detecting a SESOI of d = .50
## 0.789
## Median power for detecting a SESOI of d = .70
## 0.972
## Median power for detecting PET-PEESE estimate.PEESE estimate
## 0.609
## Median power for detecting 4/3PSM estimate.est
## 0.476Economic decision-making
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.6508614
##
## $`Publication bias`$`4/3PSM`
## est se zvalue pvalue ciLB ciUB k steps
## 0.234 0.125 1.875 0.061 -0.011 0.478 24.000 2.000
##
## $`Publication bias`$`PET-PEESE`
## PEESE estimate se zvalue pvalue ciLB
## 0.241 0.134 1.792 0.086 -0.037
## ciUB PET estimate se zvalue pvalue
## 0.519 -0.135 0.257 -0.526 0.604
## ciLB ciUB
## -0.668 0.397
##
## $`Publication bias`$`WAAP-WLS`
## method term estimate std.error statistic p.value conf.low
## 1 WAAP-WLS b0 0.3017689 0.05936583 0.05936583 0.00001090482 0.1814823
## conf.high type kAdequate
## 1 0.4220555 2 0
##
## $`Publication bias`$`p-uniform*`
## est ciLB ciUB pvalue
## 0.229290001 -0.009305462 0.461452354 0.089691375
##
## $`Publication bias`$`p-curve`
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 36
## - Total number of p<0.05 studies included into the analysis: k = 27 (75%)
## - Total number of studies with p<0.025: k = 14 (38.89%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.500 -2.332 0.010 -3.352 0
## Flatness test 0.024 -0.937 0.174 4.609 1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 21% (7.3%-43.7%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: no
##
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## Median power for detecting a SESOI of d = .20
## 0.202
## Median power for detecting a SESOI of d = .50
## 0.801
## Median power for detecting a SESOI of d = .70
## 0.976
## Median power for detecting PET-PEESE estimate.PEESE estimate
## 0.272
## Median power for detecting 4/3PSM estimate.est
## 0.260Contour enhanced funnel plots
Forest plots
## null device
## 1Moderator/sensitivity analyses
The below reported meta-regressions are all implemented as a multivariate RVE-based models using the CHE working model (Pustejovsky & Tipton, 2020; https://osf.io/preprints/metaarxiv/vyfcj/). Testing of contrasts is carried out using a robust Wald-type test testing the equality of estimates across levels of the moderator.
Moderation by citations and IF
Overall effect moderated by citations and IF
## $test
## Coef. Estimate SE t-stat p-val (z) Sig.
## 1 intrcpt 0.4209 0.0307 13.70 <0.001 ***
## 2 scale(publicationYear) -0.0334 0.0293 -1.14 0.2535
## 3 scale(citationsGSMarch2016) 0.0626 0.0309 2.03 0.0424 *
## 4 scale(h5indexGSJournalMarch2016) -0.0880 0.0403 -2.18 0.0290 *
##
## $CIs
## Coef Estimate SE d.f. Lower 95% CI
## 1 intrcpt 0.4209 0.0307 Inf 0.36071
## 2 scale(publicationYear) -0.0334 0.0293 Inf -0.09081
## 3 scale(citationsGSMarch2016) 0.0626 0.0309 Inf 0.00214
## 4 scale(h5indexGSJournalMarch2016) -0.0880 0.0403 Inf -0.16688
## Upper 95% CI
## 1 0.48116
## 2 0.02395
## 3 0.12315
## 4 -0.00903Moderation by lattitude
Overall effect moderated by lattitude
## $test
## Coef. Estimate SE t-stat p-val (z) Sig.
## 1 intrcpt 0.4584 0.0367 12.482 <0.001 ***
## 2 scale(latitudeUniversity) 0.0267 0.0362 0.737 0.461
##
## $CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.4584 0.0367 Inf 0.3864 0.5304
## 2 scale(latitudeUniversity) 0.0267 0.0362 Inf -0.0442 0.0975Compensatory effects moderated by lattitude
## $test
## Coef. Estimate SE t-stat p-val (z) Sig.
## 1 intrcpt 0.2753 0.0716 3.844 <0.001 ***
## 2 scale(latitudeUniversity) -0.0355 0.0625 -0.567 0.57
##
## $CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.2753 0.0716 Inf 0.135 0.4157
## 2 scale(latitudeUniversity) -0.0355 0.0625 Inf -0.158 0.0871Priming effects moderated by lattitude
## $test
## Coef. Estimate SE t-stat p-val (z) Sig.
## 1 intrcpt 0.5076 0.0403 12.60 <0.001 ***
## 2 scale(latitudeUniversity) 0.0589 0.0406 1.45 0.147
##
## $CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.5076 0.0403 Inf 0.4286 0.587
## 2 scale(latitudeUniversity) 0.0589 0.0406 Inf -0.0207 0.139Mood effects moderated by lattitude
## $test
## Coef. Estimate SE t-stat p-val (z) Sig.
## 1 intrcpt 0.232 0.0463 5.017 <0.001 ***
## 2 scale(latitudeUniversity) -0.016 0.0353 -0.451 0.652
##
## $CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.232 0.0463 Inf 0.1417 0.3233
## 2 scale(latitudeUniversity) -0.016 0.0353 Inf -0.0852 0.0533Moderation by gender
Overall effect moderated by gender ratio
## $test
## Coef. Estimate SE t-stat p-val (z) Sig.
## 1 intrcpt 0.427 0.0338 12.63 < 0.001 ***
## 2 scale(percFemale) 0.103 0.0388 2.64 0.00821 **
##
## $CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.427 0.0338 Inf 0.3609 0.494
## 2 scale(percFemale) 0.103 0.0388 Inf 0.0266 0.179Compensatory effects moderated by gender ratio
## $test
## Coef. Estimate SE t-stat p-val (z) Sig.
## 1 intrcpt 0.2887 0.0615 4.694 <0.001 ***
## 2 scale(percFemale) 0.0305 0.0675 0.452 0.651
##
## $CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.2887 0.0615 Inf 0.168 0.409
## 2 scale(percFemale) 0.0305 0.0675 Inf -0.102 0.163Priming effects moderated by gender ratio
## $test
## Coef. Estimate SE t-stat p-val (z) Sig.
## 1 intrcpt 0.466 0.0364 12.80 <0.001 ***
## 2 scale(percFemale) 0.144 0.0428 3.35 <0.001 ***
##
## $CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.466 0.0364 Inf 0.3944 0.537
## 2 scale(percFemale) 0.144 0.0428 Inf 0.0597 0.227Published status
## $`Model results`
## $`Model results`$test
## Coef. Estimate SE t-stat p-val (z) Sig.
## 1 factor(published)0 0.312 0.0773 4.03 <0.001 ***
## 2 factor(published)1 0.432 0.0347 12.47 <0.001 ***
##
## $`Model results`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 factor(published)0 0.312 0.0773 Inf 0.160 0.463
## 2 factor(published)1 0.432 0.0347 Inf 0.364 0.500
##
##
## $`RVE Wald test`
## test Fstat df_num df_denom p_val sig
## HTZ 2.05 1 24.3 0.164Comparing randomized and non-randomized designs
## $`Model results`
## $`Model results`$test
## Coef. Estimate SE t-stat p-val (z) Sig.
## 1 factor(rct)0 0.260 0.0543 4.79 <0.001 ***
## 2 factor(rct)1 0.447 0.0360 12.40 <0.001 ***
##
## $`Model results`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 factor(rct)0 0.260 0.0543 Inf 0.154 0.367
## 2 factor(rct)1 0.447 0.0360 Inf 0.376 0.517
##
##
## $`RVE Wald test`
## test Fstat df_num df_denom p_val sig
## HTZ 8.72 1 34.7 0.00562 **Subgroups of observational and randomized effects
Non-randomized
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 29; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0387 0.1968 24 no study
## sigma^2.2 0.0116 0.1078 29 no study/result
##
## Test for Heterogeneity:
## Q(df = 28) = 102.6511, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.2554 0.0567 4.5059 <.0001 0.1443 0.3665 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.255 0.0564 4.52 21.1 <0.001 ***
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.255 0.0564 21.1 0.138 0.373
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## -0.223 0.734
##
## $Heterogeneity
## Tau I^2
## 0.2243704 79.6290061
## Jackson's I^2 Between-cluster heterogeneity
## 89.8100000 61.2600000
## Within-cluster heterogeneity ICC
## 18.3700000 0.7700000
##
## $`Proportion of significant results`
## [1] 0.3823529
##
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.4714771
##
## $`Publication bias`$`4/3PSM`
## est se zvalue pvalue ciLB ciUB k steps
## 0.092 0.059 1.569 0.117 -0.023 0.208 24.000 2.000
##
## $`Publication bias`$`PET-PEESE`
## PET estimate se zvalue pvalue ciLB
## -0.097 0.104 -0.938 0.358 -0.312
## ciUB PEESE estimate se zvalue pvalue
## 0.118 0.094 0.061 1.536 0.139
## ciLB ciUB
## -0.033 0.222
##
## $`Publication bias`$`WAAP-WLS`
## method term estimate std.error statistic p.value conf.low conf.high
## 1 WAAP-WLS b0 0.1493071 0.09129957 0.09129957 0.2436033 -0.2435232 0.5421375
## type kAdequate
## 1 1 3
##
## $`Publication bias`$`p-uniform*`
## est ciLB ciUB pvalue
## 0.14359084 0.01474893 0.27467682 0.03523204
##
## $`Publication bias`$`p-curve`
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 23
## - Total number of p<0.05 studies included into the analysis: k = 11 (47.83%)
## - Total number of studies with p<0.025: k = 7 (30.43%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.274 -3.271 0.001 -3.501 0
## Flatness test 0.389 1.111 0.867 4.880 1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 58% (22.5%-83.7%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: no
##
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## Median power for detecting a SESOI of d = .20
## "0.367"
## Median power for detecting a SESOI of d = .50
## "0.982"
## Median power for detecting a SESOI of d = .70
## "1"
## Median power for detecting PET-PEESE estimate.PET estimate
## "ES estimate in the opposite direction"
## Median power for detecting 4/3PSM estimate.est
## "0.116"Randomized
## $`RMA results with model-based SEs`
##
## Multivariate Meta-Analysis Model (k = 144; method: REML)
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.0464 0.2155 92 no study
## sigma^2.2 0.0364 0.1907 144 no study/result
##
## Test for Heterogeneity:
## Q(df = 143) = 559.4052, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.4521 0.0364 12.4333 <.0001 0.3809 0.5234 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
## Coef. Estimate SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt 0.452 0.0364 12.4 82.9 <0.001 ***
##
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt 0.452 0.0364 82.9 0.38 0.525
##
##
## $`Prediction interval`
## 95% PI LB 95% PI UB
## -0.124 1.028
##
## $Heterogeneity
## Tau I^2
## 0.2877887 69.8305082
## Jackson's I^2 Between-cluster heterogeneity
## 80.4600000 39.1500000
## Within-cluster heterogeneity ICC
## 30.6800000 0.5600000
##
## $`Proportion of significant results`
## [1] 0.6351351
##
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.5386684
##
## $`Publication bias`$`4/3PSM`
## est se zvalue pvalue ciLB ciUB k steps
## 0.220 0.064 3.443 0.001 0.095 0.346 92.000 2.000
##
## $`Publication bias`$`PET-PEESE`
## PEESE estimate se zvalue pvalue ciLB
## 0.186 0.072 2.586 0.011 0.043
## ciUB PET estimate se zvalue pvalue
## 0.329 -0.145 0.096 -1.511 0.134
## ciLB ciUB
## -0.336 0.046
##
## $`Publication bias`$`WAAP-WLS`
## method term estimate std.error statistic p.value conf.low conf.high
## 1 WAAP-WLS b0 -0.02180643 0.1306439 0.1306439 0.8729201 -0.3414805 0.2978676
## type kAdequate
## 1 1 7
##
## $`Publication bias`$`p-uniform*`
## est ciLB ciUB pvalue
## 0.205756568 0.082334253 0.325994919 0.006398717
##
## $`Publication bias`$`p-curve`
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 140
## - Total number of p<0.05 studies included into the analysis: k = 98 (70%)
## - Total number of studies with p<0.025: k = 63 (45%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.003 -8.542 0.00 -9.642 0
## Flatness test 0.076 1.887 0.97 12.097 1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 47% (35.1%-58.7%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: no
##
##
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## Median power for detecting a SESOI of d = .20
## 0.204
## Median power for detecting a SESOI of d = .50
## 0.806
## Median power for detecting a SESOI of d = .70
## 0.977
## Median power for detecting PET-PEESE estimate.PEESE estimate
## 0.183
## Median power for detecting 4/3PSM estimate.est
## 0.237F-test of equality of variances
Mean vi for non-randomized designs
Mean vi for randomized designs
F-test
## [1] 0.1272321Comparing effects based on student and non-student samples
## $`Model results`
## $`Model results`$test
## Coef. Estimate SE t-stat p-val (z) Sig.
## 1 factor(studentSample)0 0.327 0.0456 7.16 <0.001 ***
## 2 factor(studentSample)1 0.453 0.0395 11.46 <0.001 ***
##
## $`Model results`$CIs
## Coef Estimate SE d.f. Lower 95% CI Upper 95% CI
## 1 factor(studentSample)0 0.327 0.0456 Inf 0.237 0.416
## 2 factor(studentSample)1 0.453 0.0395 Inf 0.375 0.530
##
##
## $`RVE Wald test`
## test Fstat df_num df_denom p_val sig
## HTZ 4.36 1 81 0.0398 *Year of Publication
Linear mixed-effects model. Taking into effect clustering of ESs due to originating from the same study. Using square root of variance to make the distribution normal.
## Estimate Std. Error df t value
## (Intercept) -0.1407496 0.09437155 85.97906 -1.491442
## scale(h5indexGSJournalMarch2016) -0.1697075 0.10151050 85.90369 -1.671823
## scale(publicationYear) -0.2307599 0.08925779 85.99053 -2.585320
## Pr(>|t|)
## (Intercept) 0.13950560
## scale(h5indexGSJournalMarch2016) 0.09819827
## scale(publicationYear) 0.01141363Comment: all the variables were centered for easier interpretation of model coefficients. See the negative beta for Publication Year. The higher the publication year, the lower the variance (better precision), controlling for H5.
Scatterplot year <-> precision
Size of the points indicate the H5 index (the bigger the higher) of the journal that the ES is published in.
Citations
Linear mixed-effects model. Taking into effect clustering of ESs due to originating from the same study. Using square root of variance to make the distribution normal.
## Estimate Std. Error df t value
## (Intercept) -0.201059160 0.09182019 84.97688 -2.18970532
## scale(publicationYear) -0.003047723 0.11144001 85.08921 -0.02734855
## scale(h5indexGSJournalMarch2016) -0.365198106 0.11472691 84.73581 -3.18319482
## scale(citationsGSMarch2016) 0.332488656 0.10530882 85.10522 3.15727251
## Pr(>|t|)
## (Intercept) 0.031286371
## scale(publicationYear) 0.978245772
## scale(h5indexGSJournalMarch2016) 0.002037484
## scale(citationsGSMarch2016) 0.002203503Scatterplot precision <-> citations
The relationship between precision (sqrt of variance) and number of citations.
## `geom_smooth()` using formula 'y ~ x'
H5 index
Linear mixed-effects model. Taking into effect clustering of ESs due to originating from the same study. Using square root of variance to make the distribution normal.
## Estimate Std. Error df t value
## (Intercept) -0.1141591 0.0968350 87.04277 -1.178903
## scale(h5indexGSJournalMarch2016) -0.1178194 0.1027219 86.98377 -1.146974
## Pr(>|t|)
## (Intercept) 0.2416493
## scale(h5indexGSJournalMarch2016) 0.2545376Scatterplot precision <-> journal H5
The relationship between precision (sqrt of variance) and H5 index of the journal.
Decline effect
Linear mixed-effects model. Taking into effect clustering of ESs due to originating from the same study.
## Estimate Std. Error df t value Pr(>|t|)
## (Intercept) 0.001256607 0.08446073 86.44516 0.014878 0.98816379458
## scale(sqrt(vi)) 0.361986905 0.08744592 79.13890 4.139552 0.00008628732
## scale(publicationYear) 0.114376928 0.08487961 97.53930 1.347519 0.18093479180Citation bias
Do more highly-cited studies report larger effect sizes?
## Estimate Std. Error df t value
## (Intercept) 0.22906790 0.01409808 76.12334 16.2481604
## scale(publicationYear) -0.01667034 0.01740063 82.82924 -0.9580309
## scale(h5indexGSJournalMarch2016) -0.04319160 0.01701388 64.19127 -2.5386092
## scale(citationsGSMarch2016) 0.02599613 0.01651275 85.24009 1.5743067
## Pr(>|t|)
## (Intercept) 0.00000000000000000000000001855457
## scale(publicationYear) 0.34083527493986021106309181050165
## scale(h5indexGSJournalMarch2016) 0.01356484913606161198107447063421
## scale(citationsGSMarch2016) 0.11912106790969252678724643601527P-curve for interaction effects
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 51
## - Total number of p<0.05 studies included into the analysis: k = 37 (72.55%)
## - Total number of studies with p<0.025: k = 22 (43.14%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.162 -4.955 0.000 -6.205 0
## Flatness test 0.080 0.988 0.838 7.763 1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 45% (25.8%-64.1%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: no
Counts
Simple counts: 1. How often did authors test for moderation by attachment?
##
## 0 1
## 303 19- How often did authors use which attachment measure?
##
##
## 321
## 0
## 2
## affiliation FNE scale (Leary, 1983)
## 1
## affiliation. Multi-Motive Grid (MMG) (Schmalt et al. 2000, for the English version, see Sokolowski et al. 2000)
## 1
## affiliation. five-item measure of Park and Maner (2009),
## 1
## derived from Bartholomew & Horowitz, 1991
## 1
## Fraley, Waller, & Brennan (2000)
## 14
## Fraley, Waller, & Brennan, 2000
## 1
## Need For Affiliation (Park & Maner, 2009)
## 1
## Wei, Russell, Mallinckrodt, & Vogel, 2007
## 3- Via validated tests: How often was tested for:
- Independence of awareness
##
## 0
## 323- Lack of intention
##
## 0
## 320- Efficiency of behavior
##
## 0
## 317- Lack of control of behavior
##
## 0
## 321- Via non-validated tests: How often was tested for:
- Independence of awareness
##
## 0 1
## 0.7080925 0.2167630## [1] 51- Lack of intention
##
## 0
## 323- Efficiency of behavior
##
## 0
## 320- Lack of control of behavior
##
## 0
## 322- Moderation by measures assessing attachment other than by self-disclosure of emotions
## < table of extent 0 >- Whether researchers tested for innateness
##
## 0
## 320- Has skin color/ethnicity been reported?
## < table of extent 0 >- Skin temperature location
##
## 1 2
## 6 5- Population type
##
## general special student
## 28 84 8 226- Control group type (1 = active controls)
##
## 0 1
## 6 37- In how many countries the studies were conducted and what their mean distance was from the equator
## [1] "USA" "" "China" "Portugal" "Singapore"
## [6] "Israel" "Germany" "South Korea" "Netherlands" "Japan"
## [11] "Scotland" "England" "India" "Canada" "Switzerland"
## [16] "Poland" "Italy"Number of independent studies
## [1] 84Number of papers
## [1] 33Lattitude
## Lattitude mean Lattitude SD Min Max
## 39.74729 10.84424 1.29686 57.16498